Machine Learning Equalization Techniques for High Speed PAM4 Fiber Optic Communication Systems

In this work, we apply machine learning regression and classification algorithms to the problem of equalization and detection in high-speed fiber optic communication systems. Introduction Conventional linear feed forward equalizers (FFEs) are widely used in fiber optic communication systems to mitigate inter-symbol interference resulting from a limited bandwidth of electro-optic components, as well as fiber chromatic dispersion [1]. However, linear equalization techniques (analogous to linear regression in machine learning) provide only a limited benefit for the low-cost direct detection receivers used in Ethernet applications [2,3]. Chromatic dispersion is a major transmission impairment in these systems. Dispersion acts on the electric field while the square law direct detection receivers detect intensity; all optical phase information is lost during the direct detection process. Moreover, optical amplifiers generate additive white Gaussian noise (AWGN) in the optical field, but after going through square law detection, the noise is no longer AWGN. When significant chromatic dispersion is combined with optical noise, the electrical waveform after photo-detection can be so distorted that a linear FFE is not sufficient to provide an acceptable symbol error rate. In this work, we investigate and compare several machine learning algorithms for applications in digital equalization and detection, with the goal of discovering new techniques which are more robust to non-Gaussian noise statistics, and the nonlinear distortions due to interaction of chromatic dispersion and square law detection. As with a linear FFE, the input features are taken from N consecutive samples of the received waveform (sampling rate is 2x the baud rate), with N ~ 5 to 40. At the high speeds of interest to fiber optic communication systems, the equalizer complexity and memory requirements put a major constraint on the system design. Hence, we focus on regression and classification algorithms with an eye toward practical implementation in DSP ASICs. System Model Figure 1 shows the system model and simulation setup. We simulate a 4-ary pulse amplitude modulation (PAM4) fiber optic communication system. The symbol rate is 56 Gbaud for a bitrate of 112 Gb/s; this bit rate includes native 103 GbE and overhead for FEC. The PAM4 optical modulator is driven by 2 binary electrical data streams, with each pair of bits corresponding to a PAM4 symbol taken from the alphabet {-3, -1, 1, 3}. We employ pseudo-random bit patterns (PRBS) with length of 2-1 symbols to drive the modulator. Training and test data are generated using different Fig. 1: System simulation block diagram. The highlighted block for equalizer and decision device is the focus of this paper. PRBS polynomials, and different random seeds for the noise generators. The output of the modulator propagates through a fiber optic link; the main optical channel impairments are due to fiber loss and chromatic dispersion. At the receiver, an optical amplifier boosts the signal to compensate the loss, while also adding white Gaussian noise to the optical field. An optical filter is used to reduce the optical noise entering the receiver. A photo-diode converts the optical signal to an electrical current through square law detection. The electrical signal is then sampled by an analog-to-digital converter (assumed ideal) at 2 samples per symbol. Finally, the samples are processed by a digital equalizer and a decision circuit. The equalizer and decision circuit block is the main focus of this work. The optical system simulations are performed using the commercial software package OptSim, with simulation in Matlab for the receiver digital signal processing. Figure 2 shows some simulated eye diagrams at the receiver. An eye diagram displays all the waveform symbols simultaneously by overlaying in the same plot. An ideal PAM4 eye diagram would show wide open “eyes,” as in Fig. 2 a). for the case of no noise and no dispersion. The PAM4 eyes close up after adding noise from the optical amplifier as shown in Fig. 2 b). We also observe more noise on the higher PAM4 levels; this is due to the action of square law detection on the optical field as discussed above. When both noise and significant chromatic dispersion are included in the simulation, the eyes completely close, as shown in Fig. 2 c). where we included chromatic dispersion corresponding to ~ 4km of standard single mode fiber. Although it’s hard to discern a pattern in the noise-like waveform of Fig. 2 c)., a powerful machine learning algorithms is able to detect the pattern of transmitted symbols, as shown below. Features Figure 3 a). shows the block diagram for a conventional linear equalizer based on an FIR filter structure. We can also think of the linear equalizer as a linear predictor (or linear regression), where the features are N input samples, and the linear predictive model is defined by N+1 parameters learned during training by minimizing the MSE. The output of FIR filter goes to a decision device, which makes hard decisions based on minimum Euclidean distance to the nearest PAM4 symbol. We generalize the linear equalizer to the structure shown in Figure 3 b)., where the FIR filter processing is replaced with a more general machine learning algorithm for predicting the output symbol. The input features of the machine learning algorithm are the same N samples but the algorithm may be based on either regression (followed by hard decision) or classification giving the predicted PAM4 symbols directly. Figure 4 shows a feature correlation heat map of 9 samples for a). no dispersion and b). with dispersion. The correlation heat map shows that neighbouring samples are strongly correlated due to the pulse spreading in time caused by dispersion. Hence, algorithms which rely on a statistical independence of the features, such as Naive Bayes, may not be suitable for this problem. Figure 4 c). and d). show pairwise scatter diagrams for 5 samples for the case of no dispersion, and with dispersion, respectively. The points are labelled by different colors corresponding to the received PAM4 symbols. Note the output PAM4 symbols are time aligned with the center sample. As revealed in Fig. 4 c). and d)., without dispersion, it should be easy to separate the classes; Fig. 2: Simulated eye diagrams for a). no noise, no dispersion; b). including optical noise but without chromatic dispersion; c). including both optical noise and chromatic dispersion. however, fiber dispersion tends to mix the different colored clouds together, making the regression and/or classification problem very challenging. A nonlinear regression or classification algorithm is required to separate the classes in this case. Machine Learning Algorithms All algorithms are implemented in Matlab using the Statistics and Machine Learning Toolbox, and Neural Network Toolbox. Linear regression is the baseline algorithm for comparison, as this is the simplest and most popular approach in digital equalization. The strait forward generalization of linear regression to nonlinear systems is polynomial regression, also called Volterra equalization in the communications literature [4]. For simplicity, we consider only second order Volterra equalization, where the output of the equalizer is given by h!,!(x) = θ! + θ!x[n − k] !!!